Thanks for the pull request, @MoisesGSalas!

This repository is currently maintained by @openedx/openedx-unmaintained.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

Hi @MoisesGSalas , thank you for this PR, and apologies for my late reply (I've been on leave). Will review it this week.

Thanks @pomegranited, I have not experienced that error in any instance, I only noticed it while exploring the original issue with the dashboard search.

I tagged you because I saw you were the last one to work on this and for some reason I assumed you had write permissions. Can you guide me on who could be a good last reviewer?

@MoisesGSalas

Can you guide me on who could be a good last reviewer?

Sure thing -- @Ali-Salman29 will be maintaining this repo soon. @Ali-Salman29 could you take a look at this PR for us?

@pomegranited Sure, I'll look into this.

I encountered the same issue with the Meilisearch query during testing. The query being sent to the Meilisearch API is as follows:

{
    'showRankingScore': True,
    'filter': [
        'course = "course-v1:Arbisoft+CS101+2024_T1"',
        'org = "Arbisoft"',
        'start_date <= 1734363448.717559 OR start_date NOT EXISTS',
        'NOT catalog_visibility = "none"'
    ],
    'offset': 0,
    'limit': 20
}

On debugging, I found that two indexes are being created in Meilisearch: course_info and courseware_content. However, the query is using the courseware_content index, which does not include the catalog_visibility field required by the filter.

Findings:

{'course_info': ['language', 'modes', 'org', 'catalog_visibility', 'enrollment_end'], 'courseware_content': ['_pk', 'course', 'org', 'start_date']}

• The courseware_content index contains the following fields: _pk, course, org, and start_date.
• The course_info index includes the necessary catalog_visibility field, along with other relevant fields: language, modes, org, catalog_visibility, and enrollment_end.

The query should ideally use the course_info index, as it has all the fields required by the filter. Alternatively, the courseware_content index needs to include the catalog_visibility field if it must be used for this query; as suggested in the PR.

However, it's better to first compare the indexes used for specific queries in Elasticsearch and Meilisearch. This will help us to understand if there are any discrepancies in field mappings or query behavior between the two systems.

I'll further investigate this tomorrow and share my findings here.

@MoisesGSalas

{
   "properties":{
      "content":{
         "properties":{
            "display_name":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            },
            "html_content":{
               "type":"text",
               "fields":{
                  "keyword":{
                     "type":"keyword",
                     "ignore_above":256
                  }
               }
            }
         }
      },
      "content_groups":{
         "type":"keyword"
      },
      "content_type":{
         "type":"keyword"
      },
      "course":{
         "type":"keyword"
      },
      "course_name":{
         "type":"keyword"
      },
      "id":{
         "type":"keyword"
      },
      "location":{
         "type":"keyword"
      },
      "org":{
         "type":"keyword"
      },
      "start_date":{
         "type":"date"
      }
   }
}

Both Elasticsearch and Meilisearch use the courseware_content index for this query. There are no mappings for courseware_index in the Elasticsearch mapping, but if the filter is passed, it ignores it.

For Meilisearch, the filter list must include all the queried filters. In this case, the courseware_index is passed but there wasn't any filter for it; therefore it was throwing an error.

It's recommended to add all other missing fields content_groups, content_type, and location to the filter list so that it doesn't throw an error when these values are passed in the query.

In conclusion, the PR looks good to me. I don't have writes to merge this PR but if anyone could help it would be useful.
@pomegranited

Thank you for your reviews @Ali-Salman29 and @MoisesGSalas! I don't have permission to merge here -- could one of you handle this, and tag a new release for us?

@MoisesGSalas Can you please do it for now? My onboarding is still in progress.

@Ali-Salman29, sorry I can't, I don't have write permissions on this repository.

@idegtiarov If you have some CC time available, could you help us get this merged? CC @mphilbrick211

@pomegranited, I got the write access. Should I merge the PR?